Automated stereology for determining tissue characteristics

ABSTRACT

Systems and methods for automated stereology are provided. A method can include providing an imager for capturing a Z-stack of images of a three-dimensional (3D) object; constructing extended depth of field (EDF) images from the Z-stack of images; performing a segmentation method on the EDF images including estimating a Gaussian Mixture Model (GMM), performing morphological operations, performing watershed segmentation, constructing Voronoi diagrams and performing boundary smoothing; and determining one or more stereology parameters such as number of cells in a region.

CROSS CROSS-REFERENCE TO RELATED APPLICATIONS

This application is the U.S. National Stage Application of InternationalPatent Application No. PCT/US2017/061090, filed Nov. 10, 2017, whichclaims the benefit of U.S. Provisional Patent Application Ser. No.62/420,771, filed Nov. 11, 2016, the disclosures of which are herebyincorporated by reference in its entirety, including any figures,tables, or drawings.

GOVERNMENT SUPPORT

This invention was made with government support MH076541 awarded by theNational Institutes of Health. The Government has certain rights to theinvention.

FIELD OF THE INVENTION

The present invention relates to automated stereology methods andapparatuses. More specifically, the present invention relates to methodsand apparatus for determining the characteristics of a tissue sample,including the number and size of cells.

BACKGROUND OF THE INVENTION

Unbiased stereology is used to quantify properties of higher dimensional(e.g., 3D) objects using lower dimensional (e.g., 2D) sections of theobject. Computer based stereology systems acquire data from 3Dstructures and have been developed to extract an unbiased estimation ofgeometric properties including length, area, volume, and population sizeof objects within a biological sample. Biological applications ofstereology include the unbiased estimation of a regional volume oftissue, surface area and length of cells and curvilinear fibers, and thetotal number of cells (objects of interest) in a defined reference space(region of interest).

Design-based (unbiased) stereology is the current best practice forquantifying the number of cells in a tissue sample. The majority offunding agencies, journal editors, and regulatory bodies prefer thesound mathematical basis of stereology approaches over assumption- andmodel-based methods. The major obstacle to high throughput applicationsis that current stereology approaches require time- and labor-intensivemanual data collection, which can be prohibitive on tissue samples thatinclude multiple cell types. For example, section or slice thicknessdetermination may be carried out by a user performing manual adjustmentsusing the microscope's fine focusing mechanism to locate the boundariesof slice. In addition, a user may also be required to manually locateand select objects of interest while stepping through stained tissuesections in order to perform quantitative analysis of biologicalmicrostructures. Therefore, there is a continuing need to reduce thenumber of manual steps required, as well as increase the efficiency andaccuracy of automated stereology.

BRIEF SUMMARY OF THE INVENTION

Embodiments of the present invention include systems and methods forautomated stereology. Embodiments of the present invention include anautomatic optical fractionator that can obtain accurate and efficientstereology-based estimates of the number and size of biological objects(e.g., cells) in tissue sections.

A method according to the present invention can include providing animager for capturing a Z-stack of images of a three-dimensional (3D)object, the Z-stack of images being a sequence of images of the 3Dobject captured in increments having a step size along a z-axis of the3D object; constructing extended depth of field (EDF) images from theZ-stack of images; performing a segmentation method on the EDF imagesincluding estimating a Gaussian Mixture Model (GMM), performingmorphological operations, performing watershed segmentation,constructing Voronoi diagrams and performing boundary smoothing; anddetermining one or more stereology parameters such as number of cells ina region.

An embodiment of the present invention includes a method for performingcomputerized stereology. The method can include constructing extendeddepth of field (EDF) images from the Z-stack of images; performing clumpsegmentation on the EDF images by binarizing the EDF images using athreshold determined by estimating a Gaussian Mixture Model to pixelintensities; preprocessing the EDF images by converting the EDF imagesinto grayscale and opening by reconstruction followed by closing byreconstruction; performing watershed segmentation on the EDF images,wherein regional minimas are extracted as foreground markers andboundaries between regions are used as background markers, and thewatershed segmentation is applied using the background and foregroundmakers that overlap with clumps; constructing Voronoi diagrams andsmoothing, including constructing a Voronioi map using centers offoreground regions and refining region boundaries using a Savitzy-Golayfilter, and determining one or more stereology parameters, such asnumber and size of cells in a region.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1(a)-1(i) show intermediate results of different steps insegmentation-stereology according to the present invention. FIG. 1(a)shows an original image with manual counts. FIG. 1(b) shows an EDF imageused by a segmentation method according to the present invention. FIG.1(c) shows clumps segmented using the threshold computed from anestimated Gaussian Mixture Model (GMM). FIG. 1(d) processed EDM image.

FIG. 1(e) shows regional minimas in the processed image. FIG. 1(f) showsbackground markers for watershed segmentation. FIG. 1(g) shows watershedregions reconstructed by regional minimas. FIG. 1(h) shows a Voronoidiagram produced from foreground regions in each segmented clump. FIG.1(i) shows final segmentation after smoothing region boundaries using aSavitzky-Golay filter.

FIG. 2(a) shows extended depth of field (EDF) images (right) createdfrom z-stack of images (left) with a low power objective (40×, na 0.65).

FIG. 2(b) shows extended depth of field (EDF) images (right) createdfrom z-stack of images (left) with a high power objective (100×, na1.3).

FIG. 3 is a schematic of NeuN-stained soma in a thick section at lowpower (showing over-projection on left and masking on the right) andalso shows a schematic of high signal-to-noise objects in a thicksection showing over-projection (left) and masking (right).

FIGS. 4(a)-(e) are plots of manual and automated cell counts ofdifferent tissue sections.

FIG. 5(a) shows two EDF images with variable brightness.

FIG. 5(b) shows the same two EDF images as FIG. 5(a), aftersegmentation. Additionally, the Figure shows two EDF images from ahistology section showing variable brightness.

FIGS. 6A-6D shows step-wise results of EDF image processing using anautomatic stereology framework according to the present invention. FIG.6(a) shows nucleus segmentation. FIG. 6(b) shows cell clumpsegmentation. FIG. 6(c) shows cytoplasm approximation. FIG. 6(d) showscytoplasm refinement.

FIG. 7 shows Algorithm 1, which is an automatic stereology frameworkaccording to the present invention.

FIG. 8 shows a list of equations that can be used in an automatedstereology framework according to the present invention.

FIG. 9 shows a histogram of pixel intensities of a real EDF image, itscorresponding estimated GMM and a selected threshold.

FIG. 10 shows a schematic of a coarse refinement step showing removal ofsubimages not reachable by a centroid subimage.

FIG. 11 shows a schematic of a fine refinement step according to thepresent invention.

FIG. 12 shows Corpuscle problem.

FIG. 13 shows EDF images of GFAP astrocyltes (upper) and the Iba-1microglia (lower) at high power (100× oil na 1.3).

FIG. 14 shows NeuN immunostained neurons in mouse brain section. Exampleof low (cortex; upper) and high (CA1; lower) packing density.

FIG. 15 EDF image showing neurons segmented by ASA method.

FIG. 16 shows outlined reference space on tissue sections of mousebrain.

FIG. 17 shows schematic showing stacks of z-axis images (disectorstacks) at systematic-random locations.

FIG. 18 shows results for predicted segmentation of EDF images using theCNN.

FIG. 19 shows schematic diagram of the automated stereology of theinvention. (FAST™ stands for “Fully Automatic Stereology Technology,” aphrase used to describe certain embodiments of the automated stereologyof the invention.)

FIG. 20 shows an image of a stereologer system according to anembodiment of the subject invention.

DETAILED DISCLOSURE OF THE INVENTION

Embodiments of the present invention include systems and methods forautomated stereology. Embodiments of the present invention include anautomatic optical fractionator that can obtain accurate and efficientstereology-based estimates of the number and size of biological objects(cells) in tissue sections. Used in combination with segmentationalgorithms and immunostaining methods, automatic estimates of cellnumber and size (volume) are obtainable from extended depth of fieldimages built from three-dimensional volumes of tissue (disector stacks).

Embodiments of the present invention include a novel combination ofextended depth of field (EDF) images that give 2-D representations of3-D cells in a disector volume at their optimal plane of focus, and theapplication of segmentation algorithms to these EDF images in order toautomatically make unbiased (accurate) determinations of the true numberand size (volume) of cells visualized by staining. A variety of stainingmethods can be applied, which are known in the art. By increasing thesampling stringency, the automatic estimates of cell number and sizewill approach their true value. The segmentation method can include acombination of Gaussian Mixture Model (GMM), morphological operations,watershed segmentation, Voronoi diagrams and boundary smoothing, thoughit is recognized that equivalent segmentation algorithms could achieve asimilar result. The application of a segmentation algorithm to EDFimages allows for automatic estimates of object number and size indisector volumes that represent a known fraction of a reference space,hence the designation automatic optical fractionator.

Embodiments of the present invention can include a step of nucleusdetection and segmentation. FIG. 6 shows a series of processing stepsperformed on an EDF image.

In nucleus detection and segmentation, the primary goal is to detect andsegment nuclei commonly represented by small uniform relatively dark andconvex regions. Because each segmented nucleus is an indication of acell, the result of this step directly affects the outcome of the finalcytoplasm segmentation. The three most visually distinctive andimportant features of nuclei are size, average intensity and solidity,which can be used in iterative algorithms of the present invention todetect and segment nuclei. Due to the cytoplasm segmentation methods ofthe present invention, minor segmentation inaccuracies in this step willhave only negligible effects on the final results. Finally, sincenuclear detection inaccuracy has more adverse effects on the finalsegmentation outcome, algorithms of the present invention can bedesigned to have high sensitivity to nuclei. The suggested method (oralgorithm) for this task is a novel iterative approach for detecting(and segmenting) nuclei, and the method will now be further explained.

An EDF image can first be blurred using a 2-D adaptive noise-removalfilter. An algorithm of the present invention can then iterativelybinarize the image starting with a low threshold to find seed pointsfrom different nuclei. Too small or too concave regions can be removedafter each binarization and remaining regions can be added to a nucleusmask. The nucleus mask can keep the nuclei segmented at each executionphase of the algorithm. A region replaces previous regions only if ithas greater solidity than all the previous region(s) that overlap withit. This ensures that a newly appearing region does not replace othermore convex region(s). The thresholding range can be decided based onthe minimum and maximum average intensity of a typical (or average)nucleus in the images. The image can also be iterated in multiple steps(e.g., steps of 10) for faster computation.

Two post-processing steps can also be incorporated. In the twopost-processing steps, some or all regions can be dilated and filtered.Those regions having a difference between their outer boundary averageintensity and region average intensity that is smaller than a thresholdcan be removed. It should be noted that most of the artifacts can beignored because of their size (if they are isolated) or because of theirsolidity (if they are overlapping). Although the algorithm is simple andfast, it is also very accurate on both synthetic and real image datasetsand can outperform other state-of-the-art algorithms.

FIG. 7 shows an example of an algorithm (Algorithm 1) according to thepresent invention. Filtering regions based on a maximum size can beconsidered, as seen in line 6 of Algorithm 1. The filter can increasesegmentation accuracy but should not change the results for nucleusdetection accuracy on previous cytology datasets. The nucleus,represented by region A in ground truth, is considered to be detected byregion B in the segmentation results by the rule in Equation 11 of FIG.8.

According to the present invention, clump segmentation can follownucleus detection and segmentation. In clump segmentation, the cellclumps (cellular masses that contain urothelial cells) are segmentedfrom the background. Generally, the background in each EDF image isuniformly bright and the pixels of the foreground are darker, but havemore variation. This contrast causes the brightness of the darkestbackground pixel to be intrinsically higher than the brightestforeground pixel, although this is not always the case. Therefore, asimple thresholding and some morphological operations can segment thebackground from the foreground.

According to an embodiment of the present invention, the algorithmlearns a Gaussian Mixture Model (GMM) with two components on the pixelintensities using an Expectation Maximization (EM) algorithm. OneGaussian can estimate the distribution of foreground (cell clumps) pixelintensities and the second can estimate the background pixelintensities. Using the background Gaussian distribution, the thresholdT=Q(q), where Q(.) is selected as the quantile function of the normaldistribution, which can be defined as in Equation 1 (FIG. 8), where μband b are the mean and standard deviation of the background normaldistribution and erf(.) is the error function.

FIG. 9 shows a histogram of pixel intensities of a real EDF image, itscorresponding estimated GMM and the selected threshold. After an imageis binarized using the threshold T, a connected component analysis canbe performed. Those connected components that did not contain anynucleus, or have small areas, or an average intensity greater than Q(q0)can removed. Alternatively, those nuclei that do not overlap with anysegmented cell clump can be discarded.

After clump segmentation, cytoplasm segmentation can be performed, whichinvolves segmenting the overlapping cytoplasm. Generally the best focalplane for a specific cell is found when its nucleus is in focus.Therefore, it can be safely assumed that a nucleus is in focus when itscytoplasm is also (at least relatively) in focus, and vice versa. Basedon this assumption, a cytoplasm boundary of a nucleus can beapproximated by assigning the parts of the image that have focalmeasurements that are similar to the nucleus and are relatively close.These two criteria (being relatively close to the nucleus and havingsimilar focal measurements to that of the nucleus) are the main criteriawith which to approximate the cytoplasm boundaries using the imagestack. After approximating the boundary, the boundaries can be refinedin two more steps using the EDF image.

To approximate the cytoplasm boundaries, a square grid with width W canbe overlaid on each image in the stack. Instead of assigning pixels ofthe image to different nuclei, the boundaries can be approximated byassigning grid squares (or subimages). This can increase computationalspeed and also allows for defining a focus measure to estimate the focusof the area enclosed in a grid square. Based on the above assumption, iftwo subimages that are near in distance come into focus and go out offocus similarly in different images of the image stack, then it islikely they belong to the same cell. This will give an approximation ofcytoplasm boundaries.

Considering the (i,j)-th grid square (that is in row i and column j).For image k in the stack, the focus measure of Ik ((i,j)-th grid squarein k-th image in the stack), Fk, can be defined as the standarddeviation of pixel intensities in the grid square. A focus vector of (i,j)-i,j-th grid square can be defined as the vector containing focusmeasures of all images in the stack, (F1, F2, . . . , F20) (assumingthere are 20 images in each stack in the dataset). The focus vector canthen be normalized to have values within the range [0,1] and be denotedby (F1, F2, . . . , F20).

The focus distance of the (i,j) and (i0,j0)-th grid squares, Si0,j0, canthen be defined by the i,j Euclidean distance of their correspondingnormalized focus vectors as shown in Equation 2 of FIG. 8. Equation 3shows the measure of the closeness of (i,j) and (i0,j0)-th grid squares.Finally, the likelihood of the (i,j) and (i0,j0)-th grid squaresbelonging to the same cell can be estimated by Equation 4.

Using the likelihood measure, L, defined above for two subimagesbelonging to the same cell, the likelihood of a subimage belonging tothe cytoplasm of a particular cell is estimated by considering the factthat its nucleus is part of the cell. Therefore, to find out whichsubimages are a part of a particular cell, a search is done forsubimages that have a high likelihood of belonging to the same cell withthe subimages overlapping with the nucleus. Hence, to compute thelikelihood of the (i,j)-th subimage belonging to the cytoplasm of a cellwith a nucleus that overlaps with (i1, j1), (i2, j2), . . . , (im0,jm0)-th subimages, we set m as the index of the detected nucleus in acell clump (Equation 5). Lastly, if there are N nuclei detected in acell clump, namely nucleus 1 through N, the (i,j)-th subimage can beassigned to nucleus m as shown in Equation 6. In other words, a subimageis assigned as the cytoplasm of a cell if the weighted likelihood of itbelonging to that cell is greater than the sum of the likelihoods of itbelonging to other cells in the clump. The permitted degree of overlapbetween cells in a clump can be adjusted: higher values allow the cellsin a cell clump to overlap more, and vice versa. In the next twoprocesses that are described, the approximated boundaries are refined.

The first step of refining the approximated boundaries can includecoarse refinement, which is defined as refining the boundary at thesubimage level. Unlike nuclei, which are mostly convex, the shape ofcytoplasm can show substantial concavity. Therefore, enforcing convexityon cytoplasm boundaries is not realistic, though a limited level ofconcavity can be allowed in cytoplasm boundaries. To accomplish this,reachability notation can be defined and grid squares that are notreachable from the nucleus centroid can be removed. For example, supposethat the nucleus centroid falls in the (i,j)-th grid square, it can beassumed that the (i0,j0)-th grid square is not reachable from the(i,j)-th grid square if there is at least one grid square on thediscretized line segment from (i,j) to (i0,j0) that is not assigned tothe cell. Discretization can be implemented using the fast and simplealgorithms that are known in the art (and outlined in the References,below). Removing a grid square may make previously reachable gridsquares not-reachable. Not-reachable grid squares can continue to beremoved as long as such grid squares exist. FIG. 10 shows an example oftwo removed unreachable grid squares for a cell and its final shape.

The second step of refining the approximated boundaries can include finerefinement, which refines the boundary at the pixel level. Finerefinement at the pixel level can be conducted in an iterative process.The effect of nuclei on the boundary evolution can be removed byreplacing each nucleus region's pixel intensity by the mean intensity ofits outer boundary. This operation can result in smoothing the segmentednuclei regions significantly and inhibiting edge pixels caused by nucleifrom attracting the boundaries.

Moving from a pixel outside the cell towards the centroid of its nucleuscreates a transition from a (relatively) bright to a darker pixel at thetime of entering the area of the cell (at the cytoplasm boundary). Thefirst phase of each iteration can find such locations. However, findingthe correct locations is often not an easy task because (1) these edgepixels are not always easily detectable because of low contrast andsignal to noise ratio; and (2) the presence of artifacts and non-cellscreate spurious edges. The first issue can be addressed with a filterthat smooths the transition locations using the calculated transitionlocations before and after. This step ensures that if enough edge pixelsare detected correctly, a missing/incorrectly detected edge pixel willbe recovered. To minimize the adverse effect of spurious edges in thefirst phase of each iteration, a rougher smoothing filter can be used tosmooth those values and others values further from their smoothedvalues. The filter can be applied again to remaining values and the newestimated values are used to refine the boundary. A weight vector canalso be defined to give a higher preference to edge pixels in thevicinity of the refined boundary at the previous iteration, or atapproximated boundary from previous coarse refinement step. The detailsthis step will now be discussed.

Suppose that the boundary contains pixels of coordinates (cx+rΘ cos Θ,cy+rΘ sin Θ), for Θ=0, 1, . . . , 359, where (cx, cy) are thecoordinates of the nucleus centroid. In the first iteration, for eachΘ∈{0, 1, . . . , 359}, a weight vector is defined (Equation 7) thatcontains the values of the composite of a sigmoid function with thenormalized distance of points on the radial from the boundary point. Apixel corresponded to radius Θ and stride s, p^(s)Θ has the coordinates(c_(x)+s cos Θ, c_(y)+s cos Θ). The gradient at p^(s)Θ, G(p^(s)Θ), isdefined as shown (Equation 8) where I(p) is the intensity of pixel p.For strides larger than 2rΘ and for strides smaller than 0, theintensity is respectively set to a maximum or minimum. For each Θ∈{0, 1,. . . , 359}, p^(iΘ) is selected as the edge pixel (Equation 9).

After choosing the sequence of points on the boundary, the x-coordinatescan be smoothed. To filter out the spurious edge pixels after the firstsmoothing, those pixels that have a distance greater than a thresholdfrom their smoothed estimation can be discarded. The filter can beapplied again to the remaining points and the new smoothed boundary canreplace the previous estimated boundary. This can minimize the effect ofthe spurious or inaccurately selected pixels on the boundary evolution.

FIG. 11 shows how newly selected boundary points and smoothing affectthe previous boundary of a cell in a synthetic image. The onlydifference between the first iteration and the following iterations isthat in the following iterations the strides in Equation 7 are onlyconsidered from 0 through re. Therefore, in the first iteration the areamay grow, but after that it only can shrink. Inflation in iterationsother than the first iteration can be restricted because, if there areno strong edge pixels due to poor contrast, the boundaries usuallyexpand until they reach to the cell clump boundary.

The iterations can continue until the ratio of the size ofnon-overlapping area (between the new and previous areas) to the size ofprevious area is negligible (e.g., less than 0.01). Except for a fewparameters, e.g., minimum and maximum sizes for nuclei and cytoplasm,most of the parameters in the segmentation algorithm are set in anautomatic and adaptive manner separately for each image, making theresults of the automatic framework consistent with variations in imageacquisition. An important factor that favors the accurate detection andsegmentation of cells in each image stack is that the segmentationalgorithm has been specifically designed to be resistant to lowcontrast. As part of the procedures for optimizing the presentinvention, a consistent mid-level of illumination can be determined.Because images collected in datasets will have varying brightness,intensity thresholds can be set adaptively by the estimated GMM for eachimage, allowing the algorithm to generate consistent segmentations fordifferent cell types, staining intensities and microscope settings thatcause brightness variation at the image and neuron levels underbrightfield illumination, as seen in FIG. 5.

Embodiments of the subject invention provide an automation platform forscientists, such as neuroscientists, to complete unbiased stereologystudies with greater accuracy, precision, speed, and lower costs. Insome embodiments, the automatic stereology of the invention can usemachine learning, including deep learning from a convolutional neuralnetwork (CNN) and adaptive segmentation algorithms (ASA) to segmentstained cells from EDF images created from 3-D disector volumes. Inother embodiments, the automatic stereology of the invention uses a deepbelief network (DBN), including a forward propagating network comprisingan input layer, a plurality of hidden layers, and an output layer. Whenused in neurological applications, the embodiments of the subjectinvention provide that the entire process from outlining a region ofinterest to providing results can take less than 30 minutes per brain.Compared to subjective counting with manual stereology, studies with theautomatic stereology of the invention show greater accuracy andnegligible variation from non-biological sources.

The CNN can include a convolutional layer, a Rectified Linear Unit(ReLU) layer, a pooling layer, and a fully connected (FC) layer. Theconvolution layer can comprises a plurality of filters configured todetect features of an input image. Each filter can share the same biasesand weights, and analyze the same number of input neurons. The filtercan convolve across the dimensions of the input image and compute a dotproduct of the filter and the image subset in order to generate a matrixor feature map. The convolution process can preserve the spatialrelationship between the pixels. This process can be repeated for eachfilter in the convolution layer. In order to account for real worldnon-linearity, a Rectified Linear Unit (ReLU) operation can apply anactivation function to the matrix to introduce a non-linear element tothe matrix or image, as convolution is a linear operation. In order toreduce the number of parameters and computation in the CNN, a poolinglayer can be inserted after the ReLU operation to reduce the dimensionsof each matrix or feature map. The output matrix or image of the poolinglayer can then be treated as an input image of a convolution layer. Theabove described basic steps of the CNN can be repeated to extract thedesired output. The output of the final pooling layer can be an inputfor a Fully Connected (FC) Layer. The CNN can learn to count cellsthrough the different methods including backpropagation, in which knownimages with known cell or target object counts are processed through theCNN and the accuracy or the error of the output can be recorded. If thecell number count provided by the CNN exhibits poor accuracy or higherror, parameters can be adjusted to increase the accuracy of the CNN.

In some specific neurological applications, the invention providesautomatic counts of immunostained neurons and glial cells in neocortexand CA1 of brains, such as mice and human brains. By removing manualstereology as the major obstacle to progress for many basic neuroscienceand preclinical research studies, the automated stereology of theinvention provides of novel strategies for therapeutic management ofneurological diseases and mental illnesses.

In a semi-automatic mode of the invention, automatic stereology canprovide a confirmation step following segmentation by an ASA and priorto deep learning by the CNN. The system can be additionally configuredto permit a system user to manually count cells and override a processorgenerated determination of the cell count.

Previous applications of automatic image analysis of neural elementshave focused on 2-D images on thin tissue sections. Conceptually, thisapproach is semi-quantitative because it cannot make accurate (unbiased)estimates of cell number due to sampling bias from the Corpuscle Problem(FIG. 12). The number of 2-D profiles (right) on a cut surface thatappear by a knife passing through 3-D objects (left) is not equal to thetrue number of objects (left) because of bias related to the cell size,shape and orientation. According to the disector principle, unbiasedstereology overcomes this bias by quantifying 3-D cells within a knownvolume. Similarly, the FAST approach overcomes this bias using EDFimages. The EDF algorithm captures each cell in disector volumes at itsmaximum plane of resolution and then projects each cell onto anartificial 2-D plane (see, for example, FIG. 2b ), EDF images of NeuNimmunostained neurons). As a result, the cell number on the EDF imageequals the true number of cells in the disector volume. The use ofunbiased counting rules (exclusion planes) avoids bias due to edgeeffects. Combining ASA for segmenting NeuN-stained neurons is afirst-in-class use of EDF images for unbiased stereology. Neuralnetworks have been used to solve a variety of problems and tasks indifferent industries (Speech & Image Recognition, Marketing, Retail &Sales, Banking & Finance) with a large and increasing number of imageanalysis applications to biomedical problems. Certain embodiments of theinvention provide application of a CNN to segment immunostained neurons,astrocytes, and microglia cells on high resolution EDF images forunbiased stereology of cell number. To reduce time and effort forgenerating ground truth for training the model, ASA is applied to trainthe model. For this approach the annotations are created bypreprocessing images of immunostained cells on EDF images, learning aGaussian Mixture Model on each individual image, thresholding andpost-processing the images. An end user can manually edit the segmentedimage to create a sophisticated training dataset for training the neuralnetwork. Importantly, this confirmation step in the creation of thetraining dataset will also address customer needs to interact with thedata collection process, as opposed to accepting fully automaticresults.

The number of cells within each disector can be determined and used forcalculation of total cell number using the unbiased optical fractionatormethod. According to this approach for scaling from local (disector) toregion (cortex, CA1) levels, as sampling increases the estimate ofneuron number progressively converges on the true value. Once thesampling error is sufficiently low, e.g., coefficient of error (CE) lessthan 10% (CE<0.10), the estimate will be considered stable. To achieveoptimal estimates, sampling stringencies for cells and disectors can bedetermined within each region. As such, the invention provides employinga combination of ASA/CNN to segment neural elements for stereologyanalysis.

Certain embodiments of the invention provide an effective segmentationmethod for different neural elements stained with different colorimetricprotocols and in brain regions with different packing densities. Toovercome this barrier, CNN can be used (Unet) to segment neurons thatare immunostained with high signal:noise (S:N) immunomarkers, e.g., NeuNfor neurons, and then tune this CNN to segment microglia and astrocytesstained with similarly high S:N immunostains (Iba-1 and GFAP,respectively).

In other embodiments separate ASAs are developed and optimized for eachneural element (neurons and glial cells) immunostained with definedstaining protocols. Both approaches allow for a range of pre- andpost-processing steps, leading to increased confidence that thetechnical risks can be overcome using CNN, ASAs, or a combination of thetwo.

FIG. 20 shows an example stereologer system that may be used toimplement features described above with reference to FIGS. 1-19. Thestereology system includes a microscope 100, a digital camera 110, amotorized stage 120, x, y, and z axis motors, a dual stage micrometer, adigital imaging system, a processor, a memory device, a communicationinterface connecting the microscope and a computer readable medium, acomputer 200, a high definition monitor 210, a keyboard 220, and a mouse230.

The communication interface connecting the microscope and the computerreadable medium can be, for example, a communications port, a wiredtransceiver, a wireless transceiver, and/or a network card. Thecommunication interface can be capable of communicating usingtechnologies such as Ethernet, fiber optics, microwave, xDSL (DigitalSubscriber Line), Wireless Local Area Network (WLAN) technology,wireless cellular technology, BLUETOOTH technology and/or any otherappropriate technology.

Embodiments of the stereologer system of FIG. 20 may be configured toperform any feature or any combination of features described herein. Incertain embodiments, the computer readable medium may store instructionswhich, when executed by the processor, cause the processor to performany feature or any combination of features described above.

The methods and processes described herein can be embodied as codeand/or data. The software code and data described herein can be storedon one or more machine-readable media (e.g., computer-readable media),which may include any device or medium that can store code and/or datafor use by a computer system. When a computer system and/or processerreads and executes the code and/or data stored on a computer-readablemedium, the computer system and/or processer performs the methods andprocesses embodied as data structures and code stored within thecomputer-readable storage medium.

Although FIG. 20 shows that the stereology systems includes a singlemicroscope 100, a single digital camera 110, a single motorized stage120, a single computer, 200, a single display 110. A single keyboard120, and a single mouse 230, the stereologer system may includemultiples of each or any combination of these components, and may beconfigured to perform, analogous functionality to that described herein.

It should be appreciated by those skilled in the art thatcomputer-readable media include removable and non-removablestructures/devices that can be used for storage of information, such ascomputer-readable instructions, data structures, program modules, andother data used by a computing system/environment. A computer-readablemedium includes, but is not limited to, volatile memory such as randomaccess memories (RAM, DRAM, SRAM); and non-volatile memory such as flashmemory, various read-only-memories (ROM, PROM, EPROM, EEPROM), magneticand ferromagnetic/ferroelectric memories (MRAM, FeRAM), and magnetic andoptical storage devices (hard drives, magnetic tape, CDs, DVDs); networkdevices; or other media now known or later developed that is capable ofstoring computer-readable information/data. Computer-readable mediashould not be construed or interpreted to include any propagatingsignals. A computer-readable medium of the subject invention can be, forexample, a compact disc (CD), digital video disc (DVD), flash memorydevice, volatile memory, or a hard disk drive (HDD), such as an externalHDD or the HDD of a computing device, though embodiments are not limitedthereto. A computing device can be, for example, a laptop computer,desktop computer, server, cell phone, or tablet, though embodiments arenot limited thereto.

The subject invention includes, but is not limited to, the followingexemplified embodiments.

Embodiment 1

A method for performing computerized stereology, comprising:

-   -   providing an imager for capturing a Z-stack of images of a        three-dimensional (3D) object, the Z-stack of images being a        sequence of images of the 3D object captured in increments        having a step size along a z-axis of the 3D object;    -   constructing extended depth of field (EDF) images from the        Z-stack of images;    -   performing a segmentation method on the EDF images including        estimating a Gaussian Mixture Model (GMM), performing        morphological operations, performing watershed segmentation,        constructing Voronoi diagrams and performing boundary smoothing;        and    -   determining one or more stereology parameters.

Embodiment 2

A method for performing computerized stereology, comprising:

-   -   providing an imager for capturing a Z-stack of images of a        three-dimensional (3D) object, the Z-stack of images being a        sequence of images of the 3D object captured in increments        having a step size along a z-axis of the 3D object, wherein the        3D object is a tissue sample;    -   constructing extended depth of field (EDF) images from the        Z-stack of images;    -   performing a segmentation method on the EDF images including        nucleus detection and segmentation, clump segmentation,        cytoplasm segmentation, boundary approximation, course        refinement, and fine refinement; and    -   determining one or more stereology parameters.

Embodiment 3

The method for performing computerized stereology of embodiment 2,wherein the nucleus detection and segmentation includes blurring the EDFimages using a 2-D adaptive noise-removal filter, and iterativelybinarizing the EDF images starting with a low threshold to find seedpoints from different nuclei.

Embodiment 4

The method for performing computerized stereology of according to any ofembodiments 2-3, wherein the nucleus detection and segmentation includesremoving small and concave regions after each binarization and addingremaining regions to a nucleus mask.

Embodiment 5

The method for performing computerized stereology of according to any ofembodiments 2-4, wherein the nucleus mask keeps nuclei segmented at eachexecution phase of the segmentation method.

Embodiment 6

The method for performing computerized stereology of according to any ofembodiments 2-5, wherein the clump segmentation includes learning aGaussian Mixture Model (GMM) with two components on pixel intensitiesusing an Expectation Maximization (EM) algorithm.

Embodiment 7

The method for performing computerized stereology of according to any ofembodiments 2-6, wherein a first Gaussian estimates a distribution offoreground pixel intensities and a second estimates background pixelintensities.

Embodiment 8

The method for performing computerized stereology of according to any ofembodiments 2-7, wherein cytoplasm segmentation includes approximating acytoplasm boundary of a nucleus by assigning parts of the EDF imagesthat have a focus measure similar to the nucleus and are relativelyclose.

Embodiment 9

The method for performing computerized stereology of according to any ofembodiments 2-8, wherein the course refinement includes applying a gridto the EDF images, and applying a limited level of concavity by definingreachability notation and removing grid squares that are not reachablefrom a nucleus centroid, followed by discretization.

Embodiment 10

The method for performing computerized stereology of according to any ofembodiments 2-9, wherein the fine refinement includes a pixel leveliterative process and replacing each nucleus region's pixel intensitywith a mean intensity of the nucleus region's outer boundary.

Embodiment 11

A method for performing computerized stereology, comprising:

-   -   providing an imager for capturing a Z-stack of images of a        three-dimensional (3D) object, the Z-stack of images being a        sequence of images of the 3D object captured in increments        having a step size along a z-axis of the 3D object;    -   constructing extended depth of field (EDF) images from the        Z-stack of images;    -   performing clump segmentation on the EDF images by binarizing        the EDF images using a threshold determined by estimating a        Gaussian Mixture Model to pixel intensities;    -   preprocessing the EDF images by converting the EDF images into        grayscale and opening by reconstruction followed by closing by        reconstruction;    -   performing watershed segmentation on the EDF images, wherein        regional minimas are extracted as foreground markers and        boundaries between regions are used as background markers, and        the watershed segmentation is applied using the background and        foreground makers that overlap with clumps;    -   constructing Voronoi diagrams and smoothing, including        constructing a Voronoi map using centers of foreground regions        and refining region boundaries using a Savitzy-Golay filter; and    -   determining one or more stereology parameters.

Embodiment 12

The method for performing computerized stereology of embodiment 11,wherein the clump segmentation includes segmenting clumps of regions inthe EDF images by a GMM with two components estimated based on pixelintensities using an Expectation Maximization Algorithm.

Embodiment 13

The method for performing computerized stereology of according to any ofembodiments 11-12, wherein the preprocessing includes smoothing the EDFimages and removing small dark or bright regions.

Embodiment 14

The method for performing computerized stereology of according to any ofembodiments 11-13, wherein the preprocessing includes connectingrelatively close regions and removing small region minimas.

Embodiment 15

The method for performing computerized stereology of according to any ofembodiments 11-14, wherein the foreground and background markers areregion minimas extracted from preprocessed EDF images.

Embodiment 16

The method for performing computerized stereology of according to any ofembodiments 11-15, wherein the watershed segmentation expands originalregional minimas to give a better approximation of neuron boundaries.

Embodiment 17

The method for performing computerized stereology of according to any ofembodiments 11-16, wherein the constructing Voronoi diagrams andsmoothing includes not splitting a region if the region's size is lessthan a maximum threshold and solidity of the region obtained by therefined boundary of an original region is greater than an averagesolidity of all regions.

Embodiment 18

The method for performing computerized stereology of according to any ofembodiments 11-17, wherein the constructing Voronoi diagrams andsmoothing includes not splitting a region if the region's size is lessthan a maximum threshold and solidity of the region obtained by therefined boundary of an original region is greater than an averagesolidity of all regions.

Embodiment 19

The method for performing computerized stereology of according to any ofembodiments 11-18, wherein in determining a number of cells, segmentedregions are removed that do not overlap with a region of interest oroverlap exclusion lines of a disector frame.

Embodiment 20

The method for performing computerized stereology of according to any ofembodiments 19, wherein a total number of cells (N) is determinedaccording to the following equation:Total N=[ΣQ −]·F1·F2·F3

-   -   wherein F1 is the reciprocal of the section sampling fraction        (ssf); F2 is the reciprocal of the area sampling fraction (asf);        and F3 is the reciprocal of the thickness sampling fraction        (tsf).

Embodiment 21

The method of performing computerized stereology of embodiment 20,further comprising providing a processor in operable communication witha computer-readable medium, wherein the instructions stored on thecomputer readable-readable medium, when executed, cause the processorto:

-   -   generate a three dimensional computer simulation of the        three-dimensional object;    -   generate an x-stack of sections being a sequence of sections of        the three dimensional computer simulation captured in increments        having a step size along a x-axis of the three dimensional        computer simulation; and    -   determine a number of cells contained in the three dimensional        simulation from a x-direction.

Embodiment 22

The method of performing computerized stereology of embodiment 21,further comprising

-   -   providing a processor in operable communication with a        computer-readable medium,        -   wherein the instructions stored on the computer            readable-readable medium, when executed, cause the processor            to:    -   generate a three dimensional computer simulation of the        three-dimensional object;    -   generate an y-stack of sections being a sequence of sections of        the three dimensional computer simulation captured in increments        having a step size along a y-axis of the three dimensional        computer simulation; and    -   determine a number of cells contained in the three dimensional        simulation from a y-direction.

Embodiment 23

A method for computerized stereology, the method comprising

-   -   providing a providing an imager of a Z-stack of images of a        three-dimensional (3D) object, the Z-stack of images being a        sequence of images of the 3D object captured in increments        having a step size along a z-axis of the 3D object;    -   providing a processor in operable communication with a        computer-readable medium,    -   wherein the instructions stored on the computer        readable-readable medium, when executed, cause the processor to:    -   access a deep learning structure retained in the        computer-readable medium, wherein the deep learning structured        model comprises a plurality of layers with weights and biases        assigned thereto; and        configuring the deep learning structured model to:    -   construct extended depth of field (EDF) images from the Z-stack        of images;    -   perform clump segmentation on the EDF images by binarizing the        EDF images using a threshold determined by estimating a Gaussian        Mixture Model to pixel intensities;    -   preprocess the EDF images by converting the EDF images into        grayscale and opening by reconstruction followed by closing by        reconstruction;    -   perform watershed segmentation on the EDF images, wherein        regional minimas are extracted as foreground markers and        boundaries between regions are used as background markers, and        the watershed segmentation is applied using the background and        foreground makers that overlap with clumps;    -   construct Voronoi diagrams and smoothing, including constructing        a Voronoi map using centers of foreground regions and refining        region boundaries using a Savitzy-Golay filter; and    -   determine one or more stereology parameters.

Embodiment 24

The method of embodiment 23, wherein the deep learning structure is aconvolutional neural network.

Embodiment 25

The method of embodiment 24, wherein the convolutional neural networkcomprises a plurality of convolutional layers, Rectified Linear Unit(ReLU) layers, pooling layers, and a fully connected (FC) layer.

Embodiment 26

The method according to any of the embodiments 23-25, wherein theconvolutional neural network comprises:

-   -   19 convolution layers, 4 max pooling layers, and 4 up-sampling        convolution layers.

Embodiment 27

The method of performing computerized stereology according to nay ofembodiments 23-26, further comprising:

-   -   further configuring the deep learning structure to:    -   generate a three dimensional computer simulation of the 3D        object;    -   generate an x-stack of sections being a sequence of sections of        the three dimensional computer simulation captured in increments        having a step size along a x-axis of the three dimensional        computer simulation; and    -   determine a number of cells contained in the three dimensional        simulation from a x-direction.

Embodiment 28

The method of performing computerized stereology according to any ofembodiments 23-27, further comprising:

-   -   further configuring the deep learning structure to:    -   generate a three dimensional computer simulation of the        three-dimensional object;    -   generate an y-stack of sections being a sequence of sections of        the three dimensional computer simulation captured in increments        having a step size along a y-axis of the three dimensional        computer simulation; and    -   determine a number of cells contained in the three dimensional        simulation from a y-direction.

Embodiment 29

The method of performing computerized stereology according to any ofembodiments 23-28, further comprising:

-   -   providing a processor in operable communication with a        computer-readable medium, wherein the instructions stored on the        computer readable-readable medium, when executed, cause the        processor to:    -   generate a three dimensional computer simulation of the        three-dimensional object;    -   generate a Z-stack of section being a sequence of sections of        the three dimensional computer simulated model captured in        increments having a step size along a z-axis of the three        dimensional computer simulation; and    -   determine a number of cells from a z-direction.

A greater understanding of the present invention and of its manyadvantages may be had from the following examples, given by way ofillustration. The following examples are illustrative of some of themethods, applications, embodiments and variants of the presentinvention. They are, of course, not to be considered as limiting theinvention. Numerous changes and modifications can be made with respectto the invention.

Example 1

The performance of a segmentation algorithm can be evaluated after thesegmentation ground truth is collected. Results from ASM and groundtruth were correlated along with other analytic metrics (see, forexample, Table 1, below). One of the popular measures to evaluate thesegmentation performance is the Dice Similarity Coefficient (DSC). Fortwo regions, A and B, DSC(A,B) is defined as:

$\frac{2{{A\bigcap B}}}{{A} + {B}},$where |.| is the area of the region. The Dice Similarity Coefficient(DSC) of two regions A and B is defined as shown in Equation 10 of FIG.8. Other evaluation metrics include False Negative Rate at object level(FNRo), True Positive Rate at pixel level (TPRp) and False Positive Rateat pixel level (FPRp). A segmented cell in the ground truth isconsidered missed if there is no region in the segmentation result thathas a DSC greater than 0.7. FNRo is the rate of cells missed in theground truth and TPRp and FPRp are the average of true positive andfalse positive rates, respectively, at pixel levels of those regionsthat are not missed.

From the viewpoint of algorithm segmentation, following adjustments tomaxima and minima settings, the morphological characteristics ofdifferent cells are quite similar. It is expected that the automaticstereology framework will miss less than 5% of cells on average when thepairwise cell overlapping degree is not higher than 0.3. Prior tooptimization, the algorithm is expected to detect nuclei with precisiongreater than 0.95 and recall greater than 0.90, and miss around 20% ofcells in EDF images for a Dice Similarity Coefficient less than 0.7. The20% miss rate is around half of the average miss rate reported forsubjective assessments using conventional methods for manual stereology.

Example 2

All procedures for animal handling and use were approved by the USFInstitutional Animal Care and Use Committee and followed NIH guidelinesfor the care and use of laboratory animals. Two Tg4510 male mice aged6-8 months and two age- and sex-matched non-tg littermate controls wereselected at random from the colony at the Byrd Alzheimer's Institute atthe University of South Florida in Tampa, Fla. To validate the ASF forcounting Neu-N immunostained neurons, the well-characterized Tg4510 linewas chosen with responder and activator transgenes that drive expressionof a P301L tau mutation under control of a tetracyclineoperon-responsive element. Rather than testing for a specific hypothesesrelated to tauopathies, neurodegeneration or neuroinflammation, thisline of tg mice was selected because the brains show a wide range ofneocortical cell morphologies under high power brightfield illumination,including normal and damaged neurons and resting/activated states ofneuroglia cells.

Mice were deeply anesthetized on an isothermal pad and perfused with 25ml of cold sterile buffered saline. Brains were removed and onehemisphere immersion fixed for 24 hours in freshly prepared phosphatebuffered paraformaldehyde. After fixation, brains were transferred toDulbecco's phosphate buffered saline and stored at 4° C. Prior tosectioning, brains were cryoprotected in 10, 20 and 30% sucrose. Frozen50-μm sections were collected with a sliding microtome, transferred to24 well plates in Dulbecco's phosphate buffered saline and stored at 4°C. One set of every n^(th) section was sampled in a systematic-random toobtain 6-8 sections through each neocortex.

Sampled sections were immunostained with Neu-N antibodies for high S: Nvisualization of neurons. Sections from all animals were placed in amulti-sample staining tray and endogenous peroxidase was blocked (10%methanol, 3% H₂O₂ in PBS; 30 min). Tissue samples were permeabilized(with 0.2% lysine, 1% Triton X-100 in PBS solution) and incubatedovernight in an appropriate primary antibody. Anti-NeuN (Millipore,Darmstadt, Germany) antibodies were used in this experiment. Sectionswere washed in PBS, and then incubated in corresponding biotinylatedsecondary antibody (Vector Laboratories, Burlingame, Calif.). The tissuewas again washed after 2 h and incubated with Vectastain® Elite® ABC kit(Vector Laboratories) for enzyme conjugation. Finally, sections werestained using 0.05% diaminobenzidine and 0.03% H₂0₂. Tissue sectionswere mounted onto slides, dehydrated, and cover slipped.

An algorithmic variation was developed and optimized from an ensemble ofsegmentations algorithms and Seed Detection-Region Growing approaches.The purpose of the developed algorithm was to automatically segment highS: N neurons on EDF images. The numbers of neurons within disectorvolumes was used to calculate total neuron number in a reference volumeusing the unbiased optical fractionator method [Equation 1].

Since the regions of interest (neuronal cell bodies) have arbitrarysizes, shapes, and orientations, none of these features can be assumed apriori for either the segmentation step or quantification using unbiasedstereology. The segmentation method applied was a combination ofGaussian Mixture Model (GMM), morphological operations, watershedsegmentation, Voronoi diagrams and boundary smoothing, as detailedabove. FIG. 1 shows the visual results of successive steps in thesegmentation of an EDF image. Black regions are removed due to notoverlapping with cells of interest, red regions are excluded due tooverlapping with exclusion lines, and blue regions are neuron targetsfor automated counting. Green marks are manual marks inside blueregions, yellow marks are automated marks not counted manually, and redmarks are missed manual marks. FIG. 1a shows a microscopy image with anunbiased disector frame used for manual counts. FIG. 1b is an EDF imageconstructed from the z-stack of images (disector stack) used for neuronsegmentation. The final segmentation result is illustrated in FIG. 1i ,where inclusion (green) and exclusion (red) lines shown in the originalimage are used by a manual optical disector and the automatic frameworkfor counting neurons independent of their geometric properties (size,shape, and orientation).

Clumps of regions (Neu-N neuronal cell bodies) in the image weresegmented by a Gaussian Mixture Model (GMM) with two componentsestimated based on pixel intensities using an Expectation Maximization(EM) algorithm. The image was binarized using a threshold computed by abackground Gaussian quantile function value and morphological operationsfollowed to extract the separate clumped neuron regions (FIG. 1c ).

The image was preprocessed by morphological operations with opening byreconstruction followed by closing by reconstruction. These operationssmooth the image and remove very small dark or bright regions (FIG. 1d )while connecting very close regions and removing very small regionminimas.

After preprocessing, the image foreground and background markers wereextracted for watershed segmentation. The foreground and backgroundmarkers are region minimas extracted from the preprocessed image (FIG.1e ) and boundaries between regions of a watershed segmentation (FIG. 1f), respectively. Region minimas select for neurons and remove regionsthat do not fall into previously segmented neuron clumps.

The watershed segmentation was applied using the foreground andbackground markers previously described. One of the regions correspondedto the background and the others were foreground regions. Foregroundregions that overlap with the map of segmented clumps were kept and theothers discarded (FIG. 1g ). This watershed segmentation usually expandsoriginal regional minimas and gives a better approximation of neuronboundaries. Lastly, each of the clump regions were split using theVoronoi diagrams obtained by the watershed regions within (FIG. 1h ).

In the final step, the region boundaries were refined usingSavitzky-Golay filter. This filter results in smoother boundaries andproduces less concave regions. It was observed that a region containinga single neuron may be split into two or more subregions if more thanone regional minima were detected. To diminish the adverse effect ofsuch splits, a region was not split if its size was less than a maximumthreshold and the solidity of the region obtained by the refinedboundary of original region was larger than the average solidity of allregions obtained by the refined boundaries of subregions. For the finalneuron count, segmented regions were removed that 1) do not overlap withthe region of interest; or 2) overlap the exclusion lines of thedisector frame. The number of remaining regions were chosen as thenumber of neurons that should be counted. This number summed across allsections [ΣQ⁻] was used to estimate the total number of Neu-Nimmunopositive neurons [Total N_(NeuN)] by an optical fractionatorformula:Total N _(NcuN)=[ΣQ ⁻]·F1·F2·F3where F1 is the reciprocal of the section sampling fraction (ssf); F2 isthe reciprocal of the area sampling fraction (asf); and F3 is thereciprocal of the thickness sampling fraction (tsf).

Example 3

An empirical study was carried out to determine optimal imagemagnification. Neu-N neurons were counted using manual stereology(ground truth) and the automatic framework on images collected at highpower [100× Plan Fluorite, n.a. 1.3] and low power (40× Plan Acromat,n.a. 0.65) by the following procedure. At high power, a trainedtechnician counted Neu-N neurons using the manual optical disector(ground truth) with assistance from the Stereologer system [StereologyResource Center (SRC), Tampa, Fla.]. At the first random x-y location onthe first section, Neu-N neurons were counted by thin focal-planeoptical scanning through a 10-um high disector. Before moving to thenext disector location, a stack of ten images about 1 um apart in thez-axis (so-called disector stacks) were captured and saved for analysisusing the automatic framework. This process of manual optical disectorcounting and saving disector stacks was repeated at 200 to 300systematic-random x-y locations across 7 systematically sampled sectionsthrough neocortex.

On completion, images in each disector stack were merged into a singlesynthetic Extended Depth of Field (EDF) image. Disector stacks combinedinto a single EDF image show all Neu-N neurons in focus, allowing thesegmentation algorithm to be applied to a single high power image (see,for example, FIG. 2 (lower)). The above process was repeated at lowpower (40×), and a second set of disector stacks collected and EDFimages created (see, for example, FIG. 1 (upper))]. In the analysisstep, ground truth and algorithm counts for NeuN neurons were correlatedfor the purpose of assessing whether cells magnified by low or highpower lens give superior results for the automatic framework. The lowerright panel shows disector frame and outlines of NeuN neuronsautomatically counted by thin focal plane scanning and opticalfractionator method.

There was a slightly better correlation (R²=0.95, FIG. 2 upper) at lowpower (40×) between Neu-N neuron counts for ground truth and theautomatic framework in the same disector volumes. The higher correlationfor the low power images, however, does not reflect true (accurate)numbers of NeuN neurons in each disector volume due to over-projectionand masking. As shown in FIG. 2, over-projection causes cells thatoverlap in the z-axis to be difficult to resolve as more than one cell.Also, larger cells in the z-axis can mask the presence of smaller ones,resulting in multiple cells that cannot be resolved as more than one.Both of these imaging artifacts arise from image capture using the highdepth of field 40×40 lens. This view is supported by the fact that fewerneurons were counted by the low power lens (data not shown). Incontrast, the results for Neu-N neuron counts using ground truth and theautomatic framework on disector volumes captured at high power showed aslightly lower correlation [(R2=0.90, FIG. 2 (lower)]. Over-projectionand masking artifacts in these disector volumes could be practicallyfixed by applying a modified segmentation algorithm with advancedpost-processing steps, e.g., a classifier to indicate likely split oroverlapping neurons. For images captured at low power, the sameclassifier approach could not resolve the correct number of neurons dueto the high depth of field of the low power lens. Another argument infavor of high power is that the optical fractionator method requiressection thickness measurements which are determined manually andautomatically by thin focal plane scanning through the z-axis to findthe upper and lower optical planes of each section. The high depth offield (thick focal plane) of the low power lens again inhibits preciselocalization of these optical planes. In contrast, the thin focal planeof the high power lens, which has twice the resolving power of the lowerpower lens, allows for precise localization of the upper and lowersection surfaces. To ensure counts are accurate, therefore, both groundtruth and ASF counts require high power magnification with a thin focalplane (low depth of field) objective lens.

Ground truth and automatic counts of Neu-N neurons were assessed in thesame disector volumes using the following procedure. Six to 8systematically sampled sections were prepared from each of two (2)Tg4510 mice (Tg-3, Tg-21) and two (2) non-tg controls (Ntg-2, Ntg-9).Two technicians with equivalent training and experience collected groundtruth datasets using the manual optical disector (Gundersen et al., 1988a,b). Sampling was carried out at sufficient x-y locations to achievehigh sampling stringency (CE<0.05). As detailed above, after manualoptical disector counting, disector stacks were collected in the z-axisfor neuron counts by the automatic framework.

The counts of Neu-N neurons for disector stacks analyzed by ground truthand the automated framework were summed to give the neuron counts across6 to 8 sections for each case (FIG. 4). Correlations between theautomatic framework and ground truth were assessed by the coefficient ofdetermination (R²). Analysis of variation for total number ofneocortical Neu-N neurons in the Tg4510 mice and non-tg controls wasdone by a two-tailed T-test with genotype as the independent variableand statistical significance at p<0.05.

Table 1 presents the ground truth and automated counts and correlationsfor the sum of all 85 sections analyzed for 4 different cases. Theaverage value for two data collectors was used for mouse 02 values.

TABLE 1 Neu-N neurons counts by ground truth vs. automatic stereology inthe same disectors of different 4 mice*. Collector (C) Mouse ID GroundTruth Auto. Count R² C1, C2* 02 1249 1238 >0.98 C2 21 858 878 >0.98 C103 570 603 >0.98 C1 09 558 697 >0.98 R² = correlation for manual andautomatic counts. *average counts between two collectors (C1 and C2) forthe same brain.

Correlations for ground truth and the automated framework are shown inTable 2. The correlations show uniformly close relationships betweenNeu-N neuron counts by both approaches (R²>0.98). Inter-raterreliability for ground truth was assessed by two technicians analyzingdifferent systematic-random disector locations on the same sectionsthrough brain 02 (R²=0.95; data not shown). The average value of bothdata collectors for this brain were used for comparison with resultsfrom the automatic framework.

FIGS. 4(a)-(e) are plots of manual and automated cell counts ofdifferent tissue sections. These plots of NeuN neuron counts by sectionshow relative agreement between the objective automated framework andsubjective manual counts by two data collectors (C1 and C2). Theresidual errors in these correlations arise from both approaches. FIG. 4shows plots for the manual and automated counts for each of the 5comparisons in Table 2. Results for counts of Neu-N immunostainedneurons in neocortex of Tg4510 mice and non-tg controls are shown inTable 2. Comparison of mean Neu-N neuron counts by ground truth and theautomatic framework showed a 7% difference for the non-tg mice and a 4%difference for Tg4510 mice. For the ground truth dataset, there was a24% difference in neuron number (p<0.11, ns). This difference wasslightly higher (27%) using the automatic framework, which did reachstatistical significance (p<0.04).

TABLE 2 Comparison of ground truth (manual optical disector) and theautomatic stereology framework (ASF) for total number (+/−SEM) of Neu-Nneurons in neocortex of Tg4510 mice an non-Tg controls. Ground Truth ASFMean Mean % Group N Neu-N SEM_(NeuN) Neu-N SEM_(NeuN) diff_(NeuN) Non-Tg2 1.30E+06 1.18E+5 1.39E+06 7.71E+04 +7 (n = 2) Tg4510 2 9.81E+052.76E+3 1.02E+05 1.41E+04 +4 (n = 2) % diff_(NeuN) −24 −27

Since brightness varies at the image and neuron levels under brightfieldillumination, intensity thresholds used for the segmentation step mustbe set adaptively. The GMM component of the algorithm is estimated bypixel intensities of each image separately. As shown in FIG. 5, thewhole framework is resistant to brightness variation.

This validation study showed a high correlation (R²>0.98) between theASF and ground truth for Neu-N counts. With regard to throughputefficiency, the ASF required about 30 minutes to achieve a high level ofsampling stringency (CE=0.05). In contrast, two moderately experiencedtechnicians both required about 8 times longer (about 4 hours) usingmanual stereology to estimate Neu-N number to a comparable samplingstringency on the same sections. With regard to reproducibility, asingle inter-rater comparison showed a difference of about 0.05 (95%agreement) for two technicians to analyze different samples of disectorsin a single brain. In contrast, intra- and inter-variability for the ASFby the same and different operators is negligible. Except for a fewparameters such as minimum and maximum sizes for neuron regions, most ofthe parameters in the framework are set in an automatic and adaptivemanner separately for each image, making the results of the frameworkconsistent with variations in image acquisition. Because imagescollected in the dataset had varying brightness (FIG. 5), intensitythresholds were set adaptively by the estimated GMM for each image,allowing the ASF to produce consistent segmentations for different celltypes, staining intensities and microscope settings. Despite the lowstatistical power in this study, both the ground truth and ASF showedevidence of cortical neuron loss in brains of Tg4510 mice at 6-8 monthsof age as previously reported.

In this validation study, cell counts using the automatic frameworkstrongly correlates with counts in exactly the same disector volumesusing the manual optical disector. Furthermore, this approach allows forexamination of the basis for discrepancies between the ASF and “groundtruth.” On sections with the lower ground truth counts, e.g., sections1-7 in FIG. 4 (c), the vast majority of mismatches occur when the datacollector fails to resolve overlapping neurons. The ASF according to thepresent invention handles this situation better by applying thesegmentation algorithm to split each cell at its optimal plane of focusin 3-D.

The EDF image shows each cell at its maximal plane of focus in thedisector volume. Segmentation of these profiles is required to countthose inside the disector frame and not overlapping with exclusionplanes. In addition to this purpose, segmented cell profiles are usefulfor estimating the size distribution using unbiased local sizeestimators, as has been previously reported. The incorporation of cellsize into the results further improves the framework's throughputefficiency vis-4-vis ground truth since estimating cell size requiresnegligible time and effort compared to cell number alone. By contrast,estimation of cell number and size requires twice the time and effortfor the manual stereology workflow compared to cell number alone.

The high correlation of Neu-N counts by manual and automatic approaches(Table 1) shows the framework can be practically used to automate thetime- and labor-intensive task of cell counting by unbiased stereology.The total processing time for the automatic approach was between 25 and30 minutes for counting immunostained cells in a single reference space.This time includes low-power outlining of the reference area andautomatic capture of disector stacks on each section (_18 to 20minutes), and a computation time of about 6 to 8 minutes to create EDFimages and run the algorithm. It is expected that analyzing images inRAM will decrease the analysis time per case to about 20 minutes orless.

Example 4

This example combines existing hardware for computerized stereology withsoftware driven by deep learning from a CNN. The CNN automaticallysegments immunostained neurons, astrocytes and microglial cells onimages of 3-D tissue volumes (disector stacks; for EDFs, see FIG. 2(b)for neurons, FIG. 15 for astrocytes and microglia) and make unbiasedestimates of total cell numbers using the automatic optical fractionatormethod.

The optical fractionator method can provide an unbiased stereologyestimate of cell number provided the cells can be effectively segmented.Separate adaptive ASAs can be used to segment each cell type or deeptransfer learning can be used to train a CNN to segment cells. The ASAapproach can require adjusting parameters and other customization steps(pre- and post-processing) to accurately segment cells with variablemorphologies, staining characteristics, and cell densities (FIG. 14).Alternatively, training one or more CNNs can require many person-hoursof manual annotation of EDF images to collect ground truth. Since deeplearning requires good but not extreme accuracy of input images forground truth the results from the automatic ASA can be used toautomatically train the model. Moreover, a further step is implementedto enhance the training set and address customer concerns aboutaccepting fully automatic data; i.e., automatic stereology resultswithout end user verification.

In one embodiment, a semi-automatic mode of the automated stereology isprovided. In certain such embodiments, after EDF images are segmented bythe ASA, but before training the model, the counts (clicks) on thesegmented cells will be displayed to an end user for confirmation (see,for example, FIG. 17). At this point the end user can accept theautomatically generated count or modify the count by adding counts basedon false negatives and/or removing counts from false positives. Thisstep can reduce the effort for annotating ground truth and requireminimal effort from the end user (see, for example, FIG. 17). Comparedto manual stereology counting the work is simple, non-tedious andstraightforward. End users can view and edit counts from the ASA.Typically, the time to annotate ground truth for deep learning for eachcell type per brain will be far less than the 4-5 hours of difficultcell counting for manual stereology. Also, the efforts involved inannotating ground truth for deep learning for each cell type per braincan be less than training the model by manually annotating ground truth.Once the deep learning model is trained from these annotated images,analysis of EDF image sets from the test cases will take about 10minutes for the motorized microscope stage to automatically capturedisector stacks and a few seconds for segmentation and stereology. Theresults will be a trained model from expert-validated ground truth foranalysis of subsequent test cases in 15 minute or less. As such, thisExample of the invention provides a novel technology for automaticstereology of histological sections with deep learning, and optionally,an expert input. For optimal brain related applications, the totalnumbers of three important cells (neurons, astrocytes, microglia) inbrains, for example, mouse brains, can be quantified on counterstainedtissue sections from regions with a wide range of packing densities(cortex, CA1; FIG. 14). To this end, deep learning with a CNN modeltrained by expert-validated ground truth can be generated by the ASAmethod. This approach provides sufficient cell segmentation to overcomethe technical barriers and achieves performance metrics.

The accuracy, precision, and efficiency of quantifying neural elements,e.g., cells in stained tissue sections depend on how the analysis isdone. A 2-D sampling probe (for example a knife blade) arbitrarilysamples cells with an unknown and unknowable probability related to thecell's size, shape, and orientation. Unbiased stereology can provide thetheoretical basis for avoiding this and other sampling and estimationbiases. However, the current technology with manual stereology isprohibitively time-consuming and laborious for a large and growingnumber of studies. In search of faster methods for quantifyinghistological sections, many neuroscientists have turned to less accuratemethods. The availability of automated and semi-automated microscopeslide scanners has stimulated interest in semi-quantitative imageanalysis of 2-D images at lower magnification (40× or lower). Due to theCorpuscle Problem (see, for example, FIG. 12), the total number of 3-Dcells in tissue is not equal to the total number of their 2-D profileson tissue sections (i.e., Total N3-D cells≠Total N2-D prof). Moreover, asurvey of automatic algorithms proposed to improve efficiency of cellcounting methods showed these approaches do not report cell counts butrather density (number per unit area or volume) of 2-D cell profiles.The focus on density estimates leads to data biased by artifacts fromtissue processing. Another problem is systematic underestimation ofcounts at low power due to over-projection and masking (see, forexample, FIG. 3). As illustrated in FIG. 3, the low-resolution/highdepth of field lens causes multiple objects to be counted as one. Hence,a high-resolution lens with low depth of fields is required for accuratecounts using the optical fractionator. High magnification is needed foraccurate counts due to the need to determine the section thickness,i.e., difference in distance between the upper and lower optical planesof each section, which can be done either manually or automatically. Toavoid the numerous sources of stereological bias from thesesemi-quantitative approaches, the embodiments of the subject inventionprovide alternatives to current quantitative approaches for manualstereology with automated stereology.

The automated stereology of the invention can be validated and optimizedusing the ASA/CNN approach for the populations of high S:N stained braincells of greatest clinical interest, such as neurons, astrocytes andmicroglia. These goals can be achieved by, among other things:

1. Developing standardized, high-throughput, deep learning networks forquantifying stereology parameters of neural tissues with high S:N byimmunostaining. The automated stereology method of the invention with anASA was used to quantify total number of NeuN-immunostained neurons onEDF images from mouse cerebral cortex (see, for example, Tables 3-4).These data confirm automatic stereology for total neuron number isequivalent to manual counts but with 10 times greater throughput.Comparison of both datasets with true counts from 3-D reconstruction ofdisector stacks (data not shown) revealed 20-30% more accuracy versuscurrent state-of-the-art manual stereology.

However, the same ASA did not segment neurons as well in brain regionswith high packing densities (CA1). With customization for each cell typein regions with low and high packing densities using the ASA methodmight eventually achieve similar performance as for NeuN neurons in anarea with low packing density (see, for example, FIG. 13). Rather thanattempting to overcome these technical barriers using the ASA approach,deep learning can be used to segment each cell type in brain regionswith low and high packing densities.

TABLE 3 NeuN neuron counts by manual vs. automatic stereology in thesame disectors of four mice. Collector (C) Mouse ID Manual Auto. CountR² 1, 2* 02 1249 1238 >0.96 2 21 858 878 >0.98 1 03 570 603 >0.98 1 09558 697 >0.98 R² = correlation for manual and automatic counts. *averagecounts between two collectors (C1 and C2) for the same brain

TABLE 4 Comparison of manual and automatic stereology for total number(±SEM) of NeuN neurons in neocortex of Tg4510 mice and non-Tg controls.Manual Stereology Automatic Stereology Group N Mean Neu-N SEM_(NeuN)Mean Neu-N SEM_(NeuN) % diff_(NeuN) Non-Tg (n = 2) 2 1.30E+06 1.18E+51.39E+06 7.71E+04 +7 Tg4510 (n = 2) 2 9.81E+05 2.76E+3 1.02E+06 1.41E+04+4 % diff_(NeuN) −25 −27

2) Developing automatic stereology software consistent with currentstandards of commercial programs for neuroscience research. Currently,100% of the approximately 3500 stereology studies done worldwide usecomputer assisted systems that rely on manual cell counting. Many endusers are reluctant to completely rely on automatic stereology tocollect results that are critical to their research programs. Therefore,a confirmation step is provided in certain semi-automated stereologyembodiments of the invention that allow end users to confirm or editground truth prior to training the model.

Other potential outcomes of the automated stereology of the inventionare shown in Table 5.

TABLE 5 Expected benefits of the automated stereology method of theclaimed invention. I. Increased efficiency of hypothesis testing ofbasic and preclinical research II. Cost savings for labor, time, andequipment to complete studies III. Greater time spent on productiveresearch activities IV. Faster progress toward understanding thepathology, natural history, genetics, and therapeutic management ofneurological diseases and mental illnesses V. Improved health,longevity, and quality of life.

The performance metrics for optimal performance of the automatedstereology of the invention are shown in Table 5. The performance metricfor accuracy can be assessed in comparisons to results from 3-Dreconstruction, i.e., cell counts by well-trained experts of each celltype by careful counting through disector stacks. These “gold standard”counts that give the true number of cells in each disector volume can bedone blind to results from automatic stereology.

TABLE 6 Performance Metrics ACCURACY Algorithm performance vs.Specificity (false positive): <5% 3D Gold Standard Sensitivity (falsenegative): <5% Dice Similarity Coefficient (DSC Acceptable: DSC 0.7:Poor: DSC <=0.7 (considered misses) PRECISION Inter-rater ReliabilityTest/Retest same or different user: EFFICIENCY Compared to ManualStereology Semi-automatic Mode: >10x faster Fully-automatic Mode: >20xfaster ¹Manual counts via 3D counts of cells in disector stacks (z-axisimages through a known volume) ²For NeuN neurons, GFAP astrocycles, Iba1microglia in neocortex and CA1 (hippocampus)

Stained tissue sections from male and female mice can be obtained. Forexample, stained tissue sections from normal (wild type) mice and agenetically modified mouse model (rTg4510 line) of neurodegeneration andneuroinflammation can be analyzed for the purposes of development andvalidation of the deep learning model. One set of every nth section willbe sampled in a systematic random manner to obtain 8-12 tissue sectionsfrom regions with low (neocortex) and high (CA1) packing densities.Hardware in a stereology system can include of a Leica DM2500 microscopewith low (4×), mid (40×) and high power (100×) objectives, motorizedX-Y-Z stage (Prior Electronics, Rockland, Mass.), Sony Firewire DXC-C33camera, and a PC computer. The following section gives the step-by-stepprocedure for collecting EDF images for training the CNN to segmentimmunostained brain cells in cortex and CA1 regions.

1) At low mag (4-5×) viewing of systematically sampled sections throughthe reference space, the end user outlines reference area (outlined ingreen) on the tissue section (see, for example, FIG. 16):

2) At high power (63×, oil), the automated stereology of the inventiondetermines the section thickness for calculation of reference spacevolume.

3) The automated stereology can follow a two-step process to generateEDF images from disector stacks: a) drive the motorized stage toautomatically capture stacks of z-axis images (disector stacks) at about200 systematic-random locations across x-y area of reference space forall 8-12 sections through reference volume (FIG. 17 shows 7 locationsfor 1 section). The x-y distance from one Z stack to the next isconsistently spaced (such as 200 μm apart) and the images from theprevious Z stack need not touch as there is no “stitching” involved.

Each image “slice” in the Z stack can be 1 μm thick. Tissue originallycut at 40 μm can yield 20-25 images per stack due toprocessing/shrinkage; and b) create EDF images from each disector stack.EDF image can capture all cells in the disector volume at their opticalresolution and displays them on a 2-D image.

In certain embodiments, the automated stereology of the invention canuse a deep learning architecture (Unet) neural network with 19convolution layers, 4 max pooling layers, and 4 up-sampling convolutionlayers. The input-to-input layer can use gray level images of size160*160 pixels, 27 hidden layers, and an output layer that gives binaryimage of the segmentation of size 160*160 pixels. As part of thepreliminary data, image datasets were trained, validated, and testedusing the Unet deep learning architecture. Images were cropped based onthe exclusion/inclusion lines of the disector frame (see, for example,FIG. 15) and resized to be a uniform size of 160*160 pixels. Labels ofthe EDF images (mask images) were created to show neurons as white andthe background as black (binary images). Images were augmented withrotations of 15 degrees giving 14400 and 4800 training and validationimages, respectively. A second dataset of 139 images withoutaugmentation were used as test cases. The deep learning open sourceplatform Keras (frontend) and Tensorflow neural network library(backend) were utilized. FIG. 18 shows the predicted segmentation forthe prototype software. Masks (ground truth) were created from EDFimages to represent the annotation images by experts or by the ASAmethod. Unet was used to train and validate data subsets.

The Dice coefficient for the model was 0.905. The Dice coefficient is ameasurement of similarity of two samples. In this case, the similarityof the segmentation generated from the automated stereology of theinvention was compared to the segmentation from experts (ground truth).

$\begin{matrix}{{DICE}\mspace{14mu}{coefficient}\mspace{14mu}{equation}} & \; \\{{{Dice}\mspace{14mu}{coefficent}} = {\frac{2*{{A\bigcap B}}}{{A} + {B}}.}} & {{Equation}\mspace{14mu} 1} \\{{Optical}\mspace{14mu}{fractionator}\mspace{14mu}{{formula}.}} & \; \\{{{Total}\mspace{14mu} N_{cell}} = {{\left\lbrack {{\sum Q} -} \right\rbrack \cdot F}\;{1 \cdot F}\;{2 \cdot F}\; 3.}} & {{Equation}\mspace{14mu} 2}\end{matrix}$

The Dice coefficient formula is: where |∩B| is the total number of truepositives: pixels that have intensity of “one” in both A and B, |A| isthe total number of positives in the ground truth (nonzero pixels), and|B| is total number of predicted positives: pixels appear as “one” in B.

To avoid potential edge effects for cells only partially visible on theinclusion line in the cropped EDF images, the predicted segmentationwill be overlaid on its corresponding original EDF image beforecropping. After processing the output with morphological operations toremove small regions, separate loosely connected regions, etc., FastRadial Basis Symmetry filter (of different sizes) will be used to firstdetect points inside different cells. Every detected point is then usedto morphologically reconstruct the map and all cells not intersectingexclusion lines are counted. The segmentation output of the CNN followedby the post processing steps will give the total number of each celltype in each disector volume (EDF image). For each brain, the totalnumber of each cell type (Total Ncell) will be estimated according tothe optical fractionator method, as we have recently shown. Since thesum of all disector volumes is a known fraction of the total volume ofeach region, the optical fractionator method allows for scaling from EDFimages to the total region (cortex, CA1) based on the number of cellscounted in the disector volumes for each brain as shown in Equation 2,where [ΣQ−] is the sum of cells counted in all EDF images; F1 is thereciprocal of the section sampling fraction (ssf); F2 is the reciprocalof the area sampling fraction (asf); and F3 is the reciprocal of thethickness sampling fraction (tsf).

Transfer learning is one solution that may help to segment cellsidentified by different stains and limit the number of EDF images fortraining the model. With this technique, knowledge learned from previoustrained tasks can be applied to new task in a related domain. The groundtruth data for training can be created with a combination of humanneuron segmentation and automatic segmentation. The neuron segmentationmodel can be tuned to segment Iba-1 immunostained microglia from groundtruth. As ground truth creation is tedious and time consuming, theminimal number of EDF images needed to tune the existing segmentationnetwork are determined to achieve performance metrics (Table 6). Theneuron segmentation network can then be tune to segmentGFAP-immunostained astrocytes, again with a lower requirement forlabeled training data. Due to stain variability, preprocessing can cleanthe ground truth masks of unnecessary blobs that could affect thesegmentation model. In addition, post-processing can be used to helpmasking blobs below a certain threshold, after which a morphologicaloperation for eroding and dilation could be applied to overcomevariations in staining characteristics. Therefore, transfer learning,images preprocessing, and post processing are promising tools toovercome the technical risk. It is also possible to label enough images,for example, twelve to twenty thousands, to train each segmentationsystem without transfer learning though the labeling process time willmake for slower progress. Finally, if the segmentation learned by thedeep neural network is unexpectedly inadequate, different adaptivealgorithms can be optimized for cell segmentation on EDF images.

Example 5

Dementia from Alzheimer's disease and other neurodegenerative conditionsis a significant threat to worldwide health care systems. Embodiments ofthe subject invention can create, quantify and display synapticdegeneration across whole brain maps. Whole Brain Deep LearningStereology can create extended depth of field (EDF) images from 3-Dstacks of z-axis images (disector stacks) stained for presynapticboutons through the entire brain. Segmentation and deep learning can beused on these EDF images to make automated stereology counts ofsynaptophysin-immunopositive boutons independent of regional boundariesacross the entire brain. The numbers of synapses within each disectorstack are automatically quantified for each brain and validated in theX-Y-Z planes through post-processing steps. For example, embodiments ofthe subject invention can be configured generate a three dimensionalcomputer simulation of the tissue sample from a stack of z-axis images.The three dimensional computer simulation can be segmented along thex-axis and separately along the y-axis. The 3D dimensional computersegments can be visually inspected to determine a cell count orprocessed through software. In certain embodiments, analysis softwarecan be configured to apply segmentation and deep learning techniques asdescribed herein to generate automated stereology counts from the x andy planes. The stereology counts from the x, y, and z planes can becompared to validate the cell counts. In other embodiments of thesubject invention, optical dissection can be performed from the x, y,and z planes of the tissue sample. As such, a synaptic map for brain canbe automatically generated in one hour or less with comparable accuracyto 3-D reconstruction (gold standard), which is currently prohibited forroutine studies due to the high time and labor requirement.

In certain embodiments, the invention can provide learning convolutionalneural network to automatically count synaptic boutons stained with thepresynaptic immunomarker synaptophysin. Performance testing can test foraccuracy, precision, and efficiency of automatic compared to manualstereology methods. The automated stereology of the invention canprovide greater than 95% accuracy compared to gold standard, i.e.,synapse counts by 3-D reconstruction in the same disector stacks.

A cross-sectional study can be conducted using the optimized automatedstereology of the invention on synaptophysin-immunostained tissuesections from behaviorally tested young (2-3 months) and old (6-8months) Tg4510 mice and age- and sex-matched non-tg controls. Wholebrain synaptic mapscan show synaptic degeneration across brain regionsassociated with cognitive decline.

Whole brain maps can show regionally independent areas of synapticinnervation/degeneration in treatment and control groups. Since diffusesynaptic loss is the strongest structural correlation for dementia inAlzheimer's disease and cognitive impairments in animal models, thesewhole brain synaptic maps can accelerate translation of preclinicalstudies into potential neuroprotective therapeutics and drug discoveryfor Alzheimer's disease in several ways. Whole brain synaptic maps canallow for accurate, reproducible, and high-throughput screening of leadcandidates. Since these synaptic maps include the entire brain the fullimpact of potential treatments can be identified. Automatic creation ofstercology-based synaptic maps may also eliminate the subjectiveinfluence of end-user training, experience, distractions, fatigue,motivation, etc. that currently confound No/Go decisions based onqualitative histopathology. In support of rapid, broad adoption, theautomated stereology of the invention can use motorized XYZ stage,bright-field microscope and digital camera hardware.

It should be understood that the examples and embodiments describedherein are for illustrative purposes only and that various modificationsor changes in light thereof will be suggested to persons skilled in theart and are to be included within the spirit and purview of thisapplication and the scope of the appended claims. In addition, anyelements or limitations of any invention or embodiment thereof disclosedherein can be combined with any and/or all other elements or limitations(individually or in any combination) or any other invention orembodiment thereof disclosed herein, and all such combinations arecontemplated with the scope of the invention without limitation thereto.

All patents, patent applications, provisional applications, andpublications referred to or cited herein are incorporated by referencein their entirety, including all figures and tables, to the extent theyare not inconsistent with the explicit teachings of this specification.Particularly, this specification incorporates by reference U.S. Pat. No.9,297,995, to the extent it is consistent with the teachings disclosedherein.

REFERENCES

-   [1] Benali A, Leefken I, Eysel U F, Weiler E. A computerized image    analysis system for quantitative analysis of cells in histological    brain sections. J Neurosci Meth. 2003; 125:33-43.-   [2] Bradley, A. P., Bamford, P. C. A one-pass extended depth of    field algorithm based on the over-complete discrete wavelet    transform, in: Image and Vision Computing'04 New Zealand (IVCNZ'04),    2004, pp. 279-284.-   [3] Chaudhury, B. H. Ahmady Phoulady, D. Goldgof, L. O. Hall, P. R.    Mouton, A. Hakam, and E. M. Siegel. An Ensemble Algorithm Framework    For Automated Stereology Of Cervical Cancer. In A. Petrosino,    editor, Image Analysis and Processing ICIAP, volume 8156, pages    823-832. Springer Berlin Heidelberg, 2013.-   [4] Costa L, Bollt E. A fast and accurate nonlinear spectral method    for image recognition and registration. Appl Phys Lett. 2006;    89:174102.-   [5] Gundersen, H. J. G., Bendtsen, T. F., Korbo, L., Marcussen, N.,    Møller, A., Nielsen, K., Nyengaard, J. R., Pakkenberg, B.,    Serensen, F. B., Vesterby, A., and West, M. J. Some New, Simple, and    Efficient Stereological Methods and Their Use in Pathological    Research and Diagnosis. APMIS 96: 379-394, 1988a.-   [6] Gundersen, H. J. G., Bagger, P., Bendtsen, T. F., Evans, S. M.,    Korbo, L., Marcussen, N., Moller, A., Nielsen, K., Nyengaard, J. R.,    Pakkenberg, B., Sorensen, F. B., Vesterby, A. and West, M. J. The    new stereological tools: Disector, fractionator, nucleator and point    sampled intercepts and their use in pathological research and    diagnosis. APMIS 96:857-S81 (1988b).-   [7] Ho S Y, Chao C Y, Huang H L, Chiu T W, Charoenkwan P, Hwang E.    NeurphologyJ: an automatic neuronal morphology quantification method    and its application in pharmacological discovery. BMC    Bioinformatics. 2011.-   [8] Inglis, A., Cruz, D., Roe, H E, Stanley, D L, Rosene, D.,    Urbanc, B. Automated identification of neurons and their locations.    J Microsc. 230: 339-352, 2008.-   [9] Lin G, Chawla M K, Olson K, Guzowski J, Barnes C, Roysam B.    Hierarchical, model-based merging of multiple fragments for improved    three-dimensional segmentation of nuclei. Cytometry. 2005;    63A:20-33.-   [10] Liu J Y, Ellis M, Brooke-Ball H, de Tisi J, Eriksson S H,    Brandner S, Sisodiya S M, Thornm M. High-throughput, automated    quantification of white matter neurons in mild malformation of    cortical development in epilepsy. Acta Neuropathol Commun., 13;    2:72, 2014.-   [11] Long X, Cleveland W L, Yao Y L. A new preprocessing approach    for cell recognition. IEEE T Inf Technol B. 2005; 9:407-412.-   [12] Long X, Cleveland W L, Yao Y L. Automatic detection of    unstained viable cells in bright field images using a support vector    machine with an improved training procedure. Comput Biol Med. 2006;    36:339-362.-   [13] Mouton, P. R., Gordon, M. Stereological And Image Analysis    Techniques For Quantitative Assessment Of Neurotoxicology. In    Neurotoxicology, 3rd Edition, Target Organ Toxicology Series, G.    Jean Harry, Hugh A. Tilson, (Eds), Taylor & Francis Press, London    And New York, Pp. 243-267, March 2010.-   [14] Mouton, P. R. Applications Of Unbiased Stereology To    Neurodevelopmental Toxicology, In Developmental Neurotoxicology    Research: Principles, Models, Techniques, Strategies And Mechanisms    (C. Wang And W. Slikke, Eds), John Wiley & Sons, Hoboken, Pg. 53-77,    2011a.-   [15] Mouton, P. R. Unbiased Stereology: A Concise Guide. The Johns    Hopkins University Press, Baltimore, August, 2011b.-   [16] Mouton, P. R. Quantitative Anatomy Using Unbiased Stereology,    In CRC Handbook Of Imaging In Biological Mechanics (Corey P. Neu And    Guy M. Genin, Editors), CRC Press, London, 579 Pp., Oct. 23, 2014.-   [17] Mouton, P. R., Phoulady, H. A., Goldgof, D., Hall, L. O.,    Siegel, E. Automatic Stereology Of Substantia Nigra Using A Novel    Segmentation Framework Based On The Balloon Active Contour Model.    Soc. Neurosciences, Chicago, Ill., Oct. 17, 2015.-   [18] Mouton, P. R., Phoulady, A., Hall, L. O. Goldgof, D. Tg4510    mice provide an effective model for automatic testing of    neuroprotective therapies in early-stage Alzheimer's disease. 2016    Soc. Neurosci, in press.-   [19] Nattkemper T W, Ritter H J, Schubert W. A neural classifier    enabling high-throughput topological analysis of lymphocytes in    tissue sections. IEEE T Inf Technol B. 2001; 5:138-149.-   [20] Peng, S., Urbanc, B., Cruz, L., Hyman, B. T., Stanley, H. E.    Neuron recognition by parallel Potts segmentation. Proc Natl Acad    Sci USA, 100(7):3847-52, 2003.-   [21] Phoulady, H. A., Chaudhury, B., Goldgof, D. B., Hall, L. O.,    Mouton, P. R., Hakam, A., Siegel, E. Experiments in Large Ensemble    for Segmentation and Classification of Cervical Cancer Biopsy    Images. Proceedings of IEEE International Conference on Systems,    Man, and Cybernetics, October 2014.-   [22] Phoulady, H. A., Goldgof, D., Hall, L., Mouton, P. R. An    Approach For Overlapping Cell Segmentation In Multi-Layer Cervical    Cell Volumes,” First Place In Overlapping Cervical Cytology Image    Segmentation Challenge, IEEE ISBI, Brooklyn, N.Y., March 2015.-   [23] Phoulady, H. A., Goldgof, D. B., Hall, L. O., And Mouton, P. R.    Histopathology Image Segmentation With Hierarchical Multilevel    Thresholding, Proceedings Of The SPIE Medical Imaging On Digital    Pathology, San Diego, Calif., February 2016a.-   [24] Phoulady, H. A., Goldgof, D. B., Hall, L. O., And Mouton, P. R.    A New Approach To Detect And Segment Overlapping Cells In    Multi-Layer Cervical Cell Volume Images, Proceedings Of The    International Symposium On Biomedical Imaging (ISBI), Prague, Czech    Republic, April 2016b.-   [25] Ray N, Acton S T, Ley K. Tracking leukocytes in vivo with shape    and size constrained active contours. IEEE T Med Imaging. 2002;    21:1222-1234.-   [26] Santacruz K, Lewis J, Spires T, Et Al. Tau Suppression In A    Neurodegenerative Mouse Model Improves Memory Function. Science.    2005 Jul. 15; 309(5733):476-81.-   [27] Sterio, D. C. 1984. The unbiased estimation of number and sizes    of arbitrary particles using the disector. J Microscopy.    134:127-136.-   [28] Spires, T L, Orne J D, Santacruz K, Et Al. Region-Specific    Dissociation Of Neuronal Loss And Neurofibrillary Pathology In A    Mouse Model Of Tauopathy. Am J Pathol. 168:1598-607, 2006.-   [29] Sjöström P J, Frydel B R, Wahlberg L U. Artificial neural    network-aided image analysis system for cell counting. Cytometry.    1999; 36:18-26.-   [30] Slater D, Healey G, Sheu P, Cotman C W, Su J, Wasserman A,    Shankle R. IEEE Workshop on Applications of Computer Vision (WACV)    IEEE Computer Society; Washington, D.C., USA: 1996.-   [31] Savitzky, A., Golay, M. J. Smoothing and differentiation of    data by simplified least squares procedures. Analytical Chemistry    36 (8) (1964) 1627-1639.-   [32] Valdecasas, A. F., Marshall, D., Becerra, J. M., Terrero, J. J.    On extended depth of focus algorithms for bright field microscopy.    Micron, 32:559-569, 2001.-   [33] West M J, Slomianka L, Gundersen H J. Unbiased Stereological    Estimation Of The Total Number Of Neurons In The Subdivisions Of The    Rat Hippocampus Using The Optical Fractionator. Anat Rec. 231(4):    482-97, 1991.-   [34] Wicksell, S. D. The corpuscle problem. A mathematical study of    a biometric problem. Biometrika 17: 84-89, 192.

What is claimed is:
 1. A method of performing computerized stereology,the method comprising: capturing, by an imager, a Z-stack of images of athree-dimensional (3D) object, the Z-stack of images being a sequence ofimages of the 3D object captured in increments having a step size alonga Z-axis of the 3D object; and determining, by a processor in operablecommunication with a non-transitory computer-readable medium, astereology parameter of the Z-stack of images using a deep learningstructured model, wherein the non-transitory computer-readable mediumhas instructions stored thereon that, when executed, cause the processorto use the deep learning structured model to determine the stereologyparameter of the Z-stack of images.
 2. The method according to claim 1,wherein the deep learning structured model comprises a convolutionalneural network (CNN).
 3. The method according to claim 2, wherein thedeep learning structured model further comprises an adaptivesegmentation algorithm (ASA) to segment stained cells from imagescreated from the Z-stack of images.
 4. The method according to claim 3,wherein the images created from the Z-stack of images, from which theASA segments stained cells, are extended depth of field (EDF) images. 5.The method according to claim 2, wherein the CNN comprises aconvolutional layer, a Rectified Linear Unit (ReLU) layer, a poolinglayer, and a fully connected (FC) layer.
 6. The method according toclaim 5, wherein the convolutional layer comprises a plurality offilters configured to detect features of the Z-stack of images.
 7. Themethod according to claim 6, wherein each filter of the plurality offilters has the same biases and weights, and analyzes the same number ofinput neurons, as every other filter of the plurality of filters.
 8. Themethod according to claim 7, wherein each filter convolves acrossdimensions of an image of the Z-stack of images and computes a dotproduct of the respective filter and an image subset from among theZ-stack of images to generate a matrix or a feature map, and whereineach filter preserves a spatial relationship between pixels of the imageof the Z-stack of images as it convolves across dimensions of the image.9. The method according to claim 8, wherein the ReLU layer applies anactivation function to the matrix or the feature map to introduce anon-linear element to the matrix or the feature map, and wherein thepooling layer reduces dimensions of the matrix or the feature map,generating an output matrix or an output image for the FC layer.
 10. Asystem for performing computerized stereology, the system comprising: animager configured to capture a Z-stack of images of a three-dimensional(3D) object, the Z-stack of images being a sequence of images of the 3Dobject, and the imager being configured to capture the sequence ofimages of the 3D object in increments having a step size along a Z-axisof the 3D object; a processor in operable communication with the imager;and a non-transitory computer-readable medium in operable communicationwith the processor and having instructions stored thereon that, whenexecuted, cause the processor to determine a stereology parameter of theZ-stack of images using a deep learning structured model, wherein thedeep learning structured model comprises a convolutional neural network(CNN).
 11. The system according to claim 10, wherein the deep learningstructured model further comprises an adaptive segmentation algorithm(ASA) to segment stained cells from images created from the Z-stack ofimages.
 12. The system according to claim 11, wherein the images createdfrom the Z-stack of images, from which the ASA segments stained cells,are extended depth of field (EDF) images.
 13. The system according toclaim 10, wherein the CNN comprises a convolutional layer, a RectifiedLinear Unit (ReLU) layer, a pooling layer, and a fully connected (FC)layer.
 14. The system according to claim 13, wherein the convolutionallayer comprises a plurality of filters configured to detect features ofthe Z-stack of images.
 15. The system according to claim 14, whereineach filter of the plurality of filters has the same biases and weights,and analyzes the same number of input neurons, as every other filter ofthe plurality of filters.
 16. The system according to claim 15, whereineach filter is configured to convolve across dimensions of an image ofthe Z-stack of images and compute a dot product of the respective filterand an image subset from among the Z-stack of images to generate amatrix or a feature map, and wherein each filter is configured topreserve a spatial relationship between pixels of the image of theZ-stack of images as it convolves across dimensions of the image. 17.The system according to claim 16, wherein the ReLU layer is configuredto apply an activation function to the matrix or the feature map tointroduce a non-linear element to the matrix or the feature map, andwherein the pooling layer is configured to reduce dimensions of thematrix or the feature map, generating an output matrix or an outputimage for the FC layer.
 18. A method for computerized stereology, themethod comprising: capturing, by an imager, a Z-stack of images of athree-dimensional (3D) object, the Z-stack of images being a sequence ofimages of the 3D object captured in increments having a first step sizealong a Z-axis of the 3D object; determining, by a processor in operablecommunication with a non-transitory computer-readable medium, astereology parameter of the Z-stack of images using a deep learningstructured model, wherein the non-transitory computer-readable mediumhas instructions stored thereon that, when executed, cause the processorto use the deep learning structured model to determine the stereologyparameter of the Z-stack of images, and wherein the deep learningstructured model comprises the following steps: constructing images fromthe Z-stack of images; performing clump segmentation on the constructedimages by binarizing the constructed images using a threshold determinedby estimating a Gaussian Mixture Model to pixel intensities;preprocessing the constructed images by converting the constructedimages into grayscale and opening by reconstruction followed by closingby reconstruction; performing watershed segmentation on the constructedimages, wherein regional minima are extracted as foreground markers andboundaries between regions are used as background markers, and thewatershed segmentation is applied using the background and foregroundmakers that overlap with clumps; constructing Voronoi diagrams andsmoothing, including constructing a Voronoi map using centers offoreground regions and refining region boundaries using a Savitzy-Golayfilter; and determining the stereology parameter of the Z-stack ofimages.
 19. The method according to claim 18, wherein the constructedimages are extended depth of field (EDF) images.
 20. The methodaccording to claim 19, wherein the deep learning structured modelcomprises a convolutional neural network (CNN), and wherein the CNNcomprises a convolutional layer, a Rectified Linear Unit (ReLU) layer, apooling layer, and a fully connected (FC) layer.
 21. The methodaccording to claim 19, wherein the deep learning structured modelfurther comprises the following steps: generating a 3D computersimulation of the 3D object; generating an X-stack of sections being asequence of sections of the 3D computer simulation captured inincrements having a second step size along an X-axis of the 3D computersimulation; determining a first number of cells contained in the 3Dcomputer simulation from an X-direction; generating a Y-stack ofsections being a sequence of sections of the 3D computer simulationcaptured in increments having a third step size along a Y-axis of the 3Dcomputer simulation; and determining a second number of cells containedin the 3D computer simulation from a Y-direction.